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[Title of Document] Specification 

[Title of the Invention] Controller for Photographing 
Apparatus and Photographing System 
[Scope of Claims for a Patent] 
5 [Claim 1] 

A controller for a photographing apparatus for 
controlling the photographing direction of photographing 
means that is connected through a transmission path, 
comprising: 

10 driving means for controlling the photographing 

direction of said photographing means; 

picture selecting means for controlling said 
driving means and selecting a picture displayed by said 
photographing means; 
15 a picture displaying area for driving said 

photographing means by said driving means and corresponding 
to the displayable range; and 

means for superimposing a panorama picture on said 
picture displaying area. 
20 [Claim 2] 

A controller for a photographing apparatus for 
controlling the photographing direction of photographing 
means connected through a transmission path, comprising: 
driving means for controlling the photographing 
25 direction of said photographing means; and 

a picture selecting means for controlling said 
driving means and selecting a picture displayed by said 



photographing means, 

wherein said picture selecting means comprises 
an operation area for driving said photographing means by 
saiddrivingmeansandcorrespondingtothedisplayablerange, 
5 wherein a panorama picture of said displayable 

range area superimposed and the user designates a desired 
point so as to move an object at the position corresponding 
to said desired point to a desired positional coordinates 
of said driving means in said operation area. 
10 [Claim 3] 

A controller for a photographing apparatus for 
controlling the photographing direction of photographing 
means that is connected through a transmission path, 
comprising: 

15 driving means for controlling the photographing 

direction of said photographing means; and 

picture selecting means for controlling said 
driving means and selecting a picture displayed by said 
photographing means, 

2 0 wherein said picture selecting means comprises 

an operation area for driving said photographing means by 
saiddrivingmeansandcorrespondingtothedisplayablerange, 
wherein a panorama picture in said displayable 
range is superimposed and the user designates a desired area 

25 so as to move an object at the position corresponding to 

a desiredpoint generated from saiddesiredarea to the desired 
positional coordinat es of saiddrivingmeans insaidoperat ion 



area . 

[Claim 4] 

A controller for a photographing apparatus as set 
forth in claim 1, 

wherein a current position of said driving means 
is displayed on a panorama picture in saidpicture displaying 
area . 

[Claim 5] 

A controller for a photographing apparatus as set 
forth in claim 2 or 3 , 

wherein a current position of said driving means 
is displayed on a panorama picture of said picture selecting 
means . 

[Claim 6] 

A controller for a photographing apparatus as set 
forth in claim 1, 

wherein the moving range of said driving means 
is displayed on a panorama picture of saidpicture displaying 
area . 

[Claim 7] 

A controller for a photographing apparatus as set 
forth in claim 2 or 3 , 

wherein the moving range of said driving means 
is displayed on a panorama picture of said picture selecting 
means . 

[Claim 8] 

A controller for a photographing apparatus as set 



forth in claim 1, 

wherein the angle of view of said photographing 
means is displayed on a panorama picture of said picture 
displaying area. 
[Claim 9] 

A controller for a photographing apparatus as set 
forth in claim 2 or 3 , 

wherein the angle of view of said photographing 
means is displayed on a panorama picture of said picture 
selecting means. 
[Claim 10] 

A controller for a photographing apparatus as set 
forth in claim 1, 2, or 3 , 

whereinanoperationareatowhicha current picture 
displayed by said photographing means is put is included, 
and 

wherein the user designates a desired point so 
as to move an object at the position corresponding to a desired 
point to a desired positional coordinates of said driving 
means in said operation area. 
[Claim 11] 

A controller for a photographing apparatus as set 
forth in claim 1, 2, or 3, 

wherein said picture selecting means comprises 
an operation area to which a current picture displayed by 
said photographing means is put, and 

wherein the user designates a desired area so as 



to move an object at a position corresponding to a desired 
position generated from said desired area to a desired 
positional coordinates of saiddrivingmeans insaidoperation 
area . 

[Claim 12] 

A photographing system for connecting 
photographing means and a controller for a photographing 
apparatus throughatransmissionpathandcontrolling driving 
means for controlling the photographing direction of said 
photographing means by said controller for a photographing 
apparatus, 

wherein said controller for a photographing 
apparatus comprises: 

a picture selecting means for controlling said 
driving means and selecting a picture displayed by said 
photographing means; 

a picture displaying area for driving said 
photographing means by said driving means and corresponding 
to the displayable range; and 

means for superimposing a panorama picture on said 
picture displaying area. 
[Detailed Description of the Invention] 
[0001] 

[Technical Field to which the Invention belongs] 

The present invention relates to a controller for 
a photographing apparatus and a photographing system with 
a high operational characteristic and a high visibility 



suitable for a photographing operation, using lines, of the 
apparatus that is disposed at a remote place and that is 
used for a monitoring operation, an observing operation, 
a guiding operation, a presenting operation, and so forth. 

[0002] 
[Prior Art] 

As shown in Fig. 21, when the user controls a 
photographing apparatus disposed at a remote place, he or 
she operates a pan tilter in eight directions (up, down, 
left, right, upper right, lower right, upper left, and lower 
left directions) with eight-direction keys, a zooming 
controller, and a wide-angle controller so as to photograph 
a desired object while observing a photographed picture on 
a monitor 2. Alternatively, after the user has controlled 
a photographing apparatus disposed at a remote place in the 
above -describedmethodandregisteredpan tilter information 
andzoominf ormationof posit ionsof pictures tobephotographed, 
he or she drives the photographing apparatus at absolute 
positions corresponding to the registered positions so as 
to select pictures. 
[0003] 

[Subject that the Invention is to solve] 

However, when picture and control information is 
exchanged with a photographing apparatus disposedat a remote 
place througha low- capacitynetwork, the control information 
may be lost and/or picture information may be delayed due 
to an irregularity of their arrival intervals. If the pan 



tilter or the zooming controller is operated in conventional 
method for picture and control information that has been 
delayed or lost, even if the user causes the pan tilter and 
the zooming controller to place the object at the desired 
position, the pan tilter and the zooming controller do not 
properly operate . Thus , the obj ect is placed at an improper 
position due to the delay or line condition. Thus, the user 
should control the pan tilter and the zooming controller 
basedonaprediction. Consequently, theusercannotproperly 
control the pan tilter and the zooming controller. 
[0004] 

At this time, a pan tilter and a zooming controller 
arecontrolledthroughaf eed-backoperationandanexperience . 
Therefore, when a picture loses its realtime, the amount 
of the feed-back becomes unstable. 
[0005] 

In addition, since the current picture is only 
a part that the photographing apparatus can move with the 
pan tilter, when the user detects the desired object from 
the moving range, all the moving range of the pan tilter 
is driven. In order to solve these subj ects , it was desireble 
the controlling result is predicted beforehand and the 
condition of the photographing apparatus can be easily known . 
[0006] 

Therefore, an object of the present invention is 
to provide a picture selecting means, a controller for a 
photographingapparatusandphotographingsystemf oral lowing 
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theusertoeasilyobtainthedesiredpicturewithaphotograping 
apparatus that is operated according to the picture selecting 
means . 

[0007] 

Furthermore, an object of the present invention 
is to provide a controller for a photographing apparatus 
and photographing system for allowing the user to designate 
a desired position or a desired area on a panorama picture 
displayed as a part or all the moving range of a pan tilter 
so that the user can easily obtain a desired picture with 
the photographing apparatus . 
[0008] 

[Means for solving the subject] 

The present invention described in claim 1 is a 
controller for a photographing apparatus for controlling 
the photographing direction of photographing means that is 
connected through a transmission path, comprising: driving 
means for controlling the photographing direction of said 
photographingmeans ; picture select ingmeans for controlling 
said driving means and selecting a picture displayed by said 
photographing means; a picture displaying area for driving 
said photographing means by said driving means and 
corresponding to the displayable range; and means for 
superimposing a panorama picture on said picture displaying 
area . 

[0009] 

The present invention described in claim 12 is 



a photographing system for connecting photographing means 
and a controller for a photographing apparatus through a 
transmission path and controlling driving means for 
control lingthephotographingdirectionof saidphotographing 
meansbysaidcontrollerf oraphotographingapparatus , wherein 
said controller for a photographing apparatus comprises: 
a picture selecting means for controlling said driving means 
andselectingapicturedisplayedbysaidphotographingmeans; 
a picture displaying area for driving said photographing 
meansbysaiddrivingmeansandcorrespondingtothedisplayable 
range; and means for superimposing a panorama picture on 
said picture displaying area. 
[0010] 

A picture photographed by a pan tilter camera that 
is disposed at a remote place and that can be moved in various 
directions through a ciucuit is sent to a computer. The 
picture is displayed as a panorama picture in a display area 
of a monitor. The direction of a picture selecting means 
corresponding to the direction of an object to be placed 
at the center of the angleof viewof thephotographingapparatus 
in the panorama picture is designated by a pointing device 
connected to the computer. Since the pan tilter is controlled 
with reference to the panorama picture, a desired picture 
can be photographed by the photographing apparatus . 
[0011] 

The present invention described in claim 2 is a 
controller for a photographing apparatus for controlling 
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the photographing direct ionofphotographingmeans connected 
through a transmission path, comprising: driving means for 
control lingthephotographingdirectionof saidphotographing 
means; and a picture selecting means for controlling said 
driving means and selecting a picture displayed by said 
photographing means, wherein said picture selecting means 
comprises an operation area for driving said photographing 
meansbysaiddrivingmeansandcorrespondingtothedisplayable 
range, wherein a panorama picture of said displayable range 
area superimposed and the user designates a desired point 
so as to move an object at the position corresponding to 
said desired point to a desired positional coordinates of 
said driving means in said operation area. 
[0012] 

The present invention described in claim 3 is a 
controller for a photographing apparatus for controlling 
the photographing direction of photographing means that is 
connected through a transmission path, comprising: driving 
means for controlling the photographing direction of said 
photographing means; and picture selecting means for 
controlling said driving means and selecting a picture 
displayed by said photographing means , wherein said picture 
selecting means comprises an operation area for driving said 
photographing means by said driving means and corresponding 
to the displayable range , wherein a panorama picture in said 
displayable range is superimposed and the user designates 
a desired area so as to move an object at the position 
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corresponding to a desired point generated from said desired 
area to the desired positional coordinates of said driving 
means in said operation area. 
[0013] 

A picture photographed by a pan tilter camera that 
is disposed at a remote place and that can be moved in various 
derections through a circuit is sent to a computer. The 
picture is displayed in a display area of a monitor. In 
the case that a pan tilter camera has been set up the picture 
is displayed as a panorama picture in a panorama operation 
area. A desired point to be placed at the center of the 
angle of view of the photographing apparatus in a picture 
of the operation area or the panorama operation area or 
a desired point generated with a desired area is designated 
by the pointing device connected to the computer. Thus, 
in the method of which the result is input , a selected obj ect 
canbe easilyplacedat the center of the screen. Inaddition, 
since a desired point in the operation area on the screen 
or a desiredpoint generated with a desired area is designated 
with the pointing device, the user can easily know the driving 
direction of the pan tilter camera. 
[0014] 

[Embodiment of the Invention] 

Next, withref erence to the accompanying drawings, 
embodiments of the present invention will be described . Fig . 
1 shows an external view. A monitor 2 and a mouse 8 are 
connected to a computer 1 . The computer 1 controls the driving 
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operation of a pan tiler camera 3 disposed at a remote place 
through a server 9 and a transmission path. In other words, 
a controller for the photographing apparatus is composed 
of the computer 1. As a transmission path, a communication 
line (wirelesscommunicationline, wiredcommunicationline) , 
network and the like are usable. Since the computer 1 and 
the server 9 are in the cliant relation, a plurality of the 
computer Is can be connected to the server 9. 
[0015] 

The pan tilter camera 3 is integrally composed 
of a pan tilter portion and a camera portion. In Fig. 1, 
the pan tilter camera 3 and the server 9 are disposed on 
areal sceneasdenotedby4 . Ascreenof apicturephotographed 
by the pan tilter camera 3 is denoted by 5. This screen 
is hereinafter referred to as a photographed screen. The 
photographed screen 5 is an actually photographed screen. 
When a zoom lens of the pan tilter camera 3 is placed on 
the telephotograph side, the angle of view decreases. In 
contrast, when the zoom lens of the pan tilter camera 3 is 
placed on the wide-angle side, the angle of view increases. 
[0016] 

A picture on a photographed screen 5 captured by 
a pan tilter camera 3 is sent to a computer 1 through the 
server 9 and a transmission path. The picture sent to the 
computer 1 is displayed on a monitor 2 . The monitor 2 displays 
the photographed screen 5 in an operation area 6A on the 
monitor 2 . A panorama picture including the picture 
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photographed by the pan tilter camera 3 is displayed in a 
panorama operation area 6B. Inthis exmaple, an arrow- shaped 
cursor 7 is displayed at the position of a mouse pointer 
of a mouse 8 in the panorama display area 6B, and a frame 
6C of which the mouse 8 is the center is displayed . A picture 
surrounded by the frame 6 of the panorama operation area 
6B are displayed in the operation area 6A. As described 
later, the panorama operation area 6B comprises a pan tilter 
limiter 6D, and the pan tilter limiter 6D represents the 
limit of the moving range of the pan tilter camera 3. 
[0017] 

That is , as shown in Fig. 2, the operation area 
6A and the panorama operation area 6B area displayed on the 
monitor 2 . Though not shown, a picuture that is supplyed 
from the pan t il ter camera 3 disposed at a remote place through 
the server 19 and the transmission path is displayed in the 
operation area 6A, and panorama pictures is displayed in 
the panorama operation area 6B. The panorama pictures are 
supplyed from the pan tilter camera 3, and then combined. 
[0018] 

Fig. 3 is ablockdiagramshowingthe overall system 
according to the embodiment of the present invention. The 
system shown in Fig. 3 comprises a camera portion 11, a pan 
tilter portion 12, a TV monitor 13, a server 9, a computer 

1, a pointing device 14 (such as a mouse 8) , and a monitor 

2. The pan tilter camera 3 comprises a camera portion 11 
and a pan tilter portion 12 . For example, the camera portion 
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11 is disposed on the pan tilter portion 12. The camera 
portion 11 comprises a lens block portion 15, a zoom lens 
16, a zoom portion 17, a zoom lens motor 18, a solid state 
image pickup device 19, a signal separating/automatic gain 
adjusting circuit (SH/AGC) 20, an A/D converter 21, and a 
signal processing circuit 22. The camera portion 11 
represents a video camera. 
[0019] 

Thepantilterportionl2comprisesamodecont roller 
23, a camera controller 24, a pan tilter controller 25, a 
pan motor 26, a tilt motor 27, and a pan tilter 28. The 
computer 1 comprises a controllingportion 3 1 , a video capture 
portion 29, and a storing portion 30. The video capture 
portion 29 is composed of a video capture board. 
[0020] 

Rays emitted from an object are focused to the 
solid state image pickup device 19 through a lens set and 
a diaphragm of the lens block portion 15. An example of 
the solidstate image pickup device 19 isaCCD (Charge Coupled 
Device) . The focused rays (field picture) are converted 
into a picture signal and then sent to the signal 
separating/automatic gain adjusting circuit 20 . The signal 
separating/ automat icgainadjustingcircuit2 0samples /holds 
the picture signal and controls the gain of the picture signal 
with a control signal of an auto iris (AE) . The resultant 
picture signal is sent to the signal processing circuit 22 
through the A/D converter 21 . The signal processing circuit 



22 converts the received picture signal into a brightness 
signal (Y) , a color signal (C) , and a video signal and sends 
these signals as picture signals to the video capture portion 
29 of the computer 1 through the TV monitor 13 and the server 
9 . 

[0021] 

The lens block portion 15 of the camera portion 
11 is a zoom lens that drives the zoom lens 16 and thereby 
varies the angle of view of an object to be photographed. 
The lens block portion 15 causes the zoom lens motor 18 that 
is for example a stepping motor to rotate, thereby driving 
the zoom lens 16 corresponding to a drive command received 
from the camera controller 24 of the pan tilter portion 12 . 
The camera control ler24perf orms a lens controlling operation 
(for example, focusing operation and zooming operation) , 
an exposure controlling operation (for example, diaphragm 
controlling operation, gaincontrolling operation, and speed 
controlling operation of electronic shutter) , white balance 
controlling operation, a picture quality controlling 
operation, andsof orthof the camera port ion 11 . Inaddition, 
the camera controller 24 interfaces with the mode controller 
23. As interface controlling operations with respect to 
the zoom lens 16, the camera controller 24 sends a control 
signal to the motor driver corresponding to a drive command 
of the zoom lens 16 received from the mode controller 23 
so that the zoom lens 16 is placed at the position designated 
by the command. Inaddition, the camera controllers 24 always 
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sends positional information of the zoom lens 16 to the mode 
controller 23 . 
[0022] 

The camera portion 11 is disposed on the pan tilter 
portion 12 that has a degree of freedom that are rotating 
directions of two axes of pan and tilt . The pan tilter port ion 
12 causes the pan motor 26 and the tilt motor 27 to rotate 
corresponding to a drive command received from the pan tilter 
controller 25, thereby driving a pan head and a tilt head 
of the pan tilter 28. The motors 26 and 27 are composed 
of for example stepping motors. The pan tilter controller 
25 sends a control signal to the motor drivers so that the 
panheadand the tilt headare dr iven topos i t ions corresponding 
to a pan drive command and a tilt drive command received 
from mode controller 23. In addition, the pan tilter 
controller 25 always sends positional information of the 
pan head and the tilt head to the mode controller 23. 
[0023] 

The mode controller 23 controls the overall system 
corresponding to the internal states of the camera portion 
11 and the pan tilter portion 12 and the interface information 
received from the outside of the pan tilter camera 3 as will 
be described later. The mode controller 23 is connected 
with for example the computer 1 and RS-232C interface. The 
mode controller 2 3 sends drive commands received from the 
computer 1 to the pan tilter controller 2 5 and the camera 
controller 24 so as to drive the pan tilter 2 8 and the zoom 



lens 16 of the lens block portion 15. In addition, the mode 
controller 23 sends current positional information received 
from the pan tilter controller 25 and the camera controller 
24 to the computer 1. 
[0024] 

According to the embodiment, the computer 1 is 
used toselecta picture photographed by the pan tilter camera 
3 . The computer 1 processes graphics in the operation area 
6A and the panorama operation area 6B displayed on the monitor 
2 and information of a designated position and a clicking 
operation of the pointing device 14 (mouse 8) and sends the 
resultantdatatothemodecontroller 23 . Todisplayapicture 
photographed by a camera portion 11 on the monitor 2 , a video 
capturing portion 29 is used. The video capturing portion 
2 9 allows a video signal received from the camera portion 
11 to be displayed on the monitor 2 with a desired picture 
quality. In addition, the video capturing portion 29 allows 
a picture to be captured in a particular picture format (for 
example, bit map format, still picture JPEG format, moving 
picture JPEG format, or the like) with a particular picture 
quality and to be stored in the storingport ion 3 0 (for example , 
a hard disk) of the computer 1. 
[0025] 

Next, with reference to Fig. 4, an example of a 
method for generating a panorama picture will be described. 
Now, it is assumed that the environment in the place where 
the pan tilter camera 3 is disposed is a spherical surface. 



The spherical surface is referred to as virtual spherical 
surface. In Fig. 4, two adjacent pictures on the virtual 
spherical surface are combined to one panorama picture . To 
generate a panorama picture, as shown in Fig. 4A, the pan 
tilter camera 3 disposedat the center of thespherephotographs 
two adj acent pictures on the virtual spherical surface . The 
pan tilter camera 3 photographs a plane perpendicular to 
theoptical axis of the lens thereof . Fig. 4D shows a situation 
of whichtwoadjacentpicturesonthevirtual spherical surf ace 
are photographedby the pan t i 1 1 er camera 3 and the two pictures 
are mapped to the plane perpendicular to the optical axis. 
When two adj acent pictures are simply combined, they overlap 
and distort at the overlapped portion. 
[0026] 

To prevent two adj acent pictures from overlapping 
and distorting, they are mapped to the virtual spherical 
surface as shown in Fig. 4B. Fig. 4E shows a situation of 
whichtwophotographedpicturesthatareplanesperpendicular 
totheopticalaxisaremappedtothevirtual spherical surf ace . 
In such a manner, planes perpendicular to the optical axis 
(namely, photographed pictures) are mapped to the virtual 
spherical surface . The mapped pictures are combined in such 
a manner that anoverlappedportionand anunnecessaryportion 
are removed. The picture mapped on the virtual spherical 
surface is normalized with longitude and latitude. Thus, 
a panorama picture as shown in Figs. 4C and 4D is generated. 
[0027] 



Next, a method for generating a panorama picture 
will be described. In this method, as shown in Fig. 5, one 
panoramapictureisgeneratedbycombininglOpictures . First , 
the pan tilter camera 3 (not shown) disposed at the center 
of the sphere photographs 10 pictures. At this point, as 
shown in Fig. 5A, by matching the optical axis of the lens 
of the pan tilter camera 3 to positions denoted by circles, 
the pan tilter camera 3 can obtain pictures 1 to 10. As 
shown in Fig. 5B, the pictures photographed by the pan tilter 
camera 3 are pictures on theplaneperpendicular to the optical 
axis of the lens. The obtained pictures are mapped to the 
virtual spherical surface. Thereafter, as shown in Fig. 
5C , the pictures are normalized with latitude and longitude . 
The pictures are obtained in such a manner that they are 
smoothlycombinedwithoutabreak. Thereafter, anoverlapped 
portionandunnecessaryportionareremoved. Thus, apanorama 
picture of which lOpictureare smoothly combinedis generated. 
[0028] 

Next, with reference to Fig. 6, another method 
for generating a panorama picture will be described. In 
this method, pixels obtained by the pan tilter camera 3 are 
designated to pixels of a panorama picture normalized with 
latitude and longitude (namely, coordinates (s, t) ) . As 
in the method shown in Fig. 5, when pixels of pictures 
photographed by the pan tilter camera 3 are designated to 
pixels of a panorama picture, part of pixels of the panorama 
picture may not be designated. All pixels of pictures 
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photographed by the pan tilter camera 3 should be designated 
to pixels of the panorama picture. The panorama picture 
is composed of pixels calculated for individual coordinate 
points in the following process. Angular coordinates (a, 
5 (3) (seeFig. 6B) onthevirtual spherical surf ace corresponding 

to coordinates (s, t) (see Fig. 6) of a panorama picture 
are calculated corresponding to Eq. (1) . 
[0029] 

(a, P) = (a(s) , b(t)) ... (1) 

10 (Eq. (1) will be described later with reference to Fig. 7.) 

[0030] 

As shown in Fig. 6C, coordinate data (t,,r\) of the 
obtained picture is calculated with the coordinates (s, t) , 
the angular coordinates (9, cp) of a pan tilter 28, and 
15 photographing magnification assuming that the wide edge of 

the photographing apparatus is defined as one magnification 
corresponding to Eq. (2) . 
[0031] 

(§,ti) = (f (a, p, e.cp.y), g(a, p, 9, cp,y) ) 

20 ...(2) 

(Eq. (2) will be described later with reference to Fig . 8.) 
[0032] 

Corresponding to the above -described equations, 
pixels of the panorama picture are correlated with obtained 
25 pictures so as to generate a combined picture (namely, the 

panorama picture) . 
[0033] 
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Next, with reference to Fig. 7, a method for 
converting coordinates (s, t) of a panorama picture into 
angular coordinates (a, P) on the virtual spherical surface 
will be described. In Fig. 7A, PragMin represents angular 
data at the left edge assuming that the home position of 
the pan tilter 28 is 0 (rag) . PragMax represents angular 
data at the right edge assuming that the home position of 
the pan tilter 28 is 0 (reg) . Ny 2 represents a horizontal 
coordinate of the panorama operation area 6B. -Ny 2 / 2 
represents coordinate data at the right edge of the panorama 
operation area 6B. 
[0034] 

To obtain the pan angle a with the coordinate data 
s, since the following relation is satisfied 

(PragMax - a) : (PragMax - PragMin) = (Ny 2 / 2 

- s) : Ny 2 

the pan angle a is expressed as follows. 

a = PragMax - (PragMax - PragMin) x (Ny 2 / 2 - 

s) / Ny 2 

[0035] 

In Fig. 7B, TragMin represents angular data at 
the upper edge assuming that the home position of the pan 
tilter 28 is 0 (rag) . TragMax represents angular data at 
the lower edge assuming that the home position of the pan 
tilter 28 is 0 (rag) . Nz 2 represents a vertical coordinate 
of the panorama operation area 6B . -Nz 2 / 2 represents 
coordinate data at the upper edge of the panorama operation 

24 



arae 6B. Nz 2 / 2 represents coordinate data at the lower 
edge of the panorama operation area 6B. 
[0036] 

To obtain the tilt angle Pwith the coordinate 
5 data t, since the following relation is satisfied, 

(TragMax - p) : (TragMax - TragMin) = (Nz 2 / 2 

- t) : Nz 2 

the tilt angle (3 is expressed as follows. 

P = TragMax - (TragMax - TragMin) x (Nz 2 / 2 - 

10 t) / Nz 2 

[0037] 

Next, with reference to Fig. 8, the method for 
converting a plane into a spherical surf ace will be described. 
As shown in Fig. 8A, the spatial coordinates of a point (£, 
15 ti) of a photographed picture orienting the home position 

(theoriginof latitudeandlongitude) areexpressedasf ollows . 

[0038] 
[Equation 1] 

P = e x + /r, £ e k + * 2 d e n 
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[0039] 
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At this point, the following relations are 

satisfied. 

ki = tan (X I 2y) / (Ny / 2) 

k 2 = tan (n / 2y) / (Nz / 2) 
where (Ny, Nz) represent the drive ranges (y direction and 
z direction) of the mouse pointer of the pointing device 
14 (mouse 8) ; (X, \i) represents the horizontal angle of view 
and vertical angle of view at the wide edge; and y represents 
the current zoom relative magnification (magnification 
information) assuming that the wide edge is one time (x 1) . 
[0040] 

Inaddition, asshowninFig . 8B, athree-dimensional 
rotation matrix is generally expressed as follows. 

[0041] 
[Equation 2] 

cosc{> 0 -s i ncf) 
R y (<$ = 0 1 0 
sin<|> 0 cosc{> 



RAQ) 



cos8 -sine 0 
sine cose 0 
0 0 1 



[0042] 

Since the direction of one point (§, T|) of a 
photographed picture that is panned and tilted by angular 
information (0, (p) from the home position is the same as the 
direction of one point (a, P) apart from the home position, 
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the following relation is satisfied. 
[0043] 

R z (9) R y (<p) p = 1R Z (a) R y (p) e x 
When the formula is solved with respect to p, the 
5 following relation is satisfied. 

[0044] 
[Equation 3] 

p = I R y (-d>) R 2 (a - 0) R y (3) e x 

cos (a - 0) cos<t>cos£ + sin(|>sin3 

= / sin (a - 0) cosp 

. -cos (a - 0) s i n<t> cosfi + cos<t> s i n(3 



[0045] 

Thereon, 
[0046] 
[Equation 4] 



(3) 



[0047] 

Thus, ^ and T] are obtained as follows. 
[0048] 

1 = 1 /a 

% = -lb / k x = -b / k la 
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ti = lc / k 2 = c / k 2a 
With the above formula, (£, t\) projected to the photograph 
coordinates can be obtained with coordinate data with an 
angle (a, (3) from the home position. 
5 [0049] 

(-sin (a-0) cos P) / (ki (cos (a-9) cos (p cos 
P + sin cp sin P) ) 

r\ = (-cos (a-0) sin (p cos P + cos cp sin P) / (k 2 
(cos (a-8) cos (p cos P + sin (p sin P) ) 
10 Coordinate data (£, r\) on the obtained picture by the pan 

tilter camera 3 can be obtained from angular coordinates 
(a, P) on the virtual spherical surface corresponding to 
coordinate (s, t) of a panorama picture. Thus, a panorame 
picture can be generated. 
15 [0050] 

In contrast, coordinate data with an angle (a, 
P) can be obtained with (£,, r|) projected to photograph 
coordinates corresponding to the following formula. 
[0051] 

20 Since 1 = |p | 

a = 1 / V (1 + kx 2 ^ 2 + k 2 2 r| 2 ) 
b = -k x § / V (1 + k^ 2 + k 2 Y) 
c = k 2 t| / V (1 + k x 2 4 2 + k 2 2 Ti 2 ) 
where V ( ) represents that the square root of the calculated 
25 result in ( ) is obtained. 

[0052] 

Form Formula (3) , the following relations are 
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satisfied. 

a = cos(a-G) cos <p cos (3 + sin cp sin P 
b = sin (a-0) cos P 

c = -cos (a-0) sin 9 cos P + cos (p sin P 
[0053] 

Thus, the following relations are satisfied. 

a sin cp + c sin 0 = sin P 

tan (a-0) = b / (a cos q> - c sin 0) 
Thus, the following relations are satisfied. 

p = sin" 1 (sin (p / V (1 + k x 2 % + k 2 2 rf) + sin 0 
k 2 n / V(l + kx 2 % + k 2 2 if) 

a = tan" 1 (-ki \ / (cos <p - k 2 r\ sin 0) ) + 0 
[0054] 

Thus, the pan angle a and the tilt angle P can 
be obtained as follows. 

(a, P) = (f (§,ti, 9,9,7) , g(£.Ti, 0, cp, y) ) 
... (4) 

[0055] 

If an error is permitted to some extent, (a, P) 
can be expressed as follows. 
[0056] 

' a = 0 + a/7) x / N y) 
P = 9+ (ji/y) x (Ti / Nz) 
In other words, Eq. (4) can be simplified as follows, 
(a, P) = (f (4, 0,7), g(Ti,q>,y)) ... (5) 
[0057] 

Next, with reference to Fig. 9, a method for 



calculating angular information (a, P) of the pan tilter 28 
expressed by Eq. (4) andEq. (5) with positional coordinates 
(£, T|) of the operation area 6A will be described. First of 
all , an example of a method for directly designating a desired 
5 point in the operation area 6A will be described. Assuming 

that the center of the operation area 6A is defined as (0, 
0) of relative coordinates as shown in Fig . 9A, the positional 
coordinates (§, r\) of the mouse pointer of the mouse 8 in 
the operation area 6A are obtained. 

10 [0058] 

Next, another method for designating a desired 
point generated with a desired area in the operation area 
6A will be described. As shown in Fig. 9A, after a start 
point (ml, nl) in a desired area is designated, an end point 

15 (m2 , n2 ) inthedesiredarea isdesignated. As the coordinates 

at the center of therectanglegeneratedwiththese twopoints , 
a desired point T|) is obtained as Eq. (6) . 

(^,ti) = ((ml, nl) + (m2, n2)) / 2 ... (6) 

[0059] 

2 0 Fig. 9A shows coordinates of the mouse 8 (pointing 

device 14) in the operation area 6A. In Fig. 9A, the moving 
range (y direction and z direction) of the mouse pointer 
of the mouse 8 in the operation area 6A is denoted by (Ny x , 
Nzi) . Angular coordinates (a, P) of the pan tilter 28 are 

25 obtained with positional coordinates (£, T|) of the desired 

point (atthemousepointerof themouse8) , angular information 
(9, (p) that represents the orientation of the pan tilter 28, 
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andmagnif ication information (y) of the current zoomrelative 
magnification assuming that the wide edge of the zoom lens 
16 is defined as one magnification corresponding to Eq. (4) 
or Eq. (5) . 
[0060] 

The angular coordinates (a, P) shown in Fig. 9B 
are used to place a posit ion designated by the pointing device 
to the center of the photographed screen assuming that the 
home position of the pan tilter 28 is defined as the origin 
of latitude and longitude. 
[0061] 

The coordinates obtained in Fig . 9 may be absolute 
coordinates of the screen of the monitor 2 or relative 
coordinates assuming that the center of the operation area 
6A is defined as (0, 0) . In the coordinates shown in Fig. 
9, coordinates in the pan direction are represented by 
ml, m2 , 0, and a and coordinates in the tilt direction are 
represented by T|, nl, n2 , (p, and p. 
[0062] 

Thus, when the mouse pointer of the mouse 8 is 
present in the operation area 6A, the angular information 
(a, P) of the pan tilter 28 is calculated with the angular 
information (9, cp) of the current pan tilter 28 obtained with 
received data, the zoom magnification information (y) , and 
the positional information T|) at the mouse pointer of 
the mouse 8 corresponding to Eq. (4) or Eq. (5) so that the 
designated object is placed at the center of the operation 



area 6A. The angular coordinates (a, P) of the pan tilter 
2 8 are converted into internal positional information (p-new, 
t-new) asshowninFig.il. The resultant internal positional 
information (p-new, t-new) is stored in a send buffer along 
with an absolute position drive command of the pan tilter 
28. In addition, as will be described later, a data send 
requestf lag (Flag-so) issetsothatdataissentuponoccurrence 
of a timer event. 
[0063] 

Next, with reference to Fig. 10, a method for 
converting positional coordinates (§, T|) of the mouse pointer 
of the mouse 8 in the panorama operat ion area 6B of the panorama 
picture into angular coordinates (a, P) corresponding to the 
present invention will be described. As with the method 
for directly designating a desired point in the operation 
area 6A, as shown in Fig. 10A, with a method for directly- 
designating a desired point in the panorama operation area 
6B, positional coordinates (£, T|) at the mouse pointer of 
the mouse 8 can be obtained. 
[0064] 

Next, another method for designating a desired 
point generated with a desired area in the panorama operation 
area 6B will be described. As shown in Fig. 10A, after a 
start point (ml, nl) of a desired area is designated, the 
end point (m2, n2) of the desired area are designated. 
Corresponding to Eq. (6), a desired point r\) is obtained. 
[0065] 



In Fig. 10A, the moving range (y direction and 
z direction) of the mouse pointer of the mouse 8 in the panorama 
operationarea6B (themovingrangeisdef inedasthecoordinates 
of the mouse pointer of the mouse 8 (pointing device 14) 
in the panorama operation area 6B) is represented by (Ny 2 , 
Nz 2 ) . The moving range is limited by the pan filter limiter 
6D denoted by dotted lines in the panorama operation area 
6B . The pan tilter limiter 6D represents the moving range 
of the optical axis of the lens of the pan tilter camera 
3. In other words, a point cannot be designated out of the 
pan tilter limiter 6D. Positional coordinates (x, y) in 
the panorama operation area 6B, angle-of -view information 
(s, t) , and angular information (a, P) of the pan tilter 28 
can be obtained with the positional coordinates (£, r\) of 
thedesiredpoint , theangular information (9, cp) representing 
the orientation of the pan tilter 28, and the magnification 
information (y) as the current zoom relative magnification 
assuming that the wide edge of the zoom lens 16 is defined 
as one magnification corresponding to Eq. (7) , Eq. (8) , and 
Eq. (9) . 

[0066] 

(x, y) = (f o (0) , g 0 (q>) ) • • • (7) 
(s, t) = (fi(y) , gi (y) ) ... (8) 
(o,P) = (f (^) , g(ri)) ... (9) 

InFig. 10B,positionalcoordinates (x,y) represent 
the current orientation of the pan tilter 28 assuming that 
the home position of the pan tilter 28 is defined as the 
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originof latitude and longitude . Angle-of -viewinf ormation 
(s, t) is the current angle of view in the operation area 
6A. Fig. 10B represents the states of the zoom lens and 
the pan tilter in the panorama operation area 6B. 
[0067] 

In Fig. 10C, angular coordinates (a, P) are used 
to place the position designated by the pointing device to 
the center of the photographed screen assuming that the home 
position of the pan tilter 28 is defined as the origin of 
latitude and longitude. (PragMax, TragMax) and (PragMin, 
TragMin) represent the movingrange of the pantilter (namely, 
the range represented by the pan tilter limiter 6D) . Fig. 
10C shows a drive target value in the pan tilter moving range . 
[0068] 

InFig . 10 , coordinatestobeobtainedmaybeabsolute 
coordinates on the screen of the monitor 2 or relative 
coordinates assuming that the center of thepanorama operation 
area 6B is defined as (0, 0) . In the coordinates , coordinates 
in the pan direction are represented by ml, m2 , x, s, 
and a and coordinates in the tilt direction are represented 
by r|, nl, n2, y, t, and p. 
[0069] 

Thus, when the mouse pointer of the mouse 8 is 
present in thepanorama operationarea6B, angular information 
(a, p) of the pan tilter 28 is calculated with positional 
information r|) at the mouse pointer of the mouse 8 
corresponding to Eq. (9) so that the designated object in 



the operation area 6A is placed at the center of the operation 
area 6A. Angular coordinates (a, P) of the pan tilter 2 8 are 
convertedintointernalpositionalinformation (p-new, t-new) 
of the pan tilter 28 corresponding to the method shown in 
Figs. HAandllB. The internalpositional information (p-new, 
t-new) of the pan tilter 28 is stored in a send buffer along 
with an absolute position drive command of the pan tilter 
28. In addition, as will be described later, a data send 
requestf lag (Flag-so) issetsothatdataissentuponoccurrence 
of the timer event. 
[0070] 

Next, a method for converting internal positional 
information (p, t) of thepantilter28intoangularinf ormation 
(9, (p) and a method for converting angular coordinates (a, 
P) into internal positional information (p-new, t-new) of 
the pan tilter 28 will be described with reference to Fig. 
11. In Fig. 11A, PragMin represents angular data at the 
left edge assuming that the home position of the pan tilter 
28 is 0 (rag) . PragMax represents angular data at the right 
edge assuming that the home position of the pan tilter 28 
is 0 (rag) . PdatMin represents internal count data at the 
left edge of the pan tilter controller 25 . PdatMax represents 
internal counter data at the right edge of the pan tilter 
controller 25. 
[0071] 

To obtain the pan angle 9 with the pan data p, 
since the following relation is satisfied, 



(PragMax - 6) : (PragMax - PragMin) = (PdatMax 

- p) : (PdatMax - PdatMin) 

the pan angle 0 is expressed as follows. 

9 = PragMax - (PragMax - PragMin) x (PdatMax - 
p) / (PdatMax - PdatMin) 
[0072] 

Thus, the pan data p is expressed as follows, 
p = PdatMax - (PragMax - 0) x (PdatMax - PdatMin) 
/ (PragMax - PragMin) 
[0073] 

In addition, to obtain the pan data p-new with 
the pan angle a, since the following relation is satisfied, 
(PragMax - a) : (PragMax - PragMin) = (PdatMax 

- p-new) : (PdatMax - PdatMin) 

the pan data p-new is expressed as follows. 

P-new = PragMax - (PragMax -a) x (PdatMax - PdatMin) 
/ (PragMax - PragMin) 
[0074] 

In Fig. 11B, TragMin represents angular data at 
the upper edge assuming that the home position of the pan 
tilter 28 is 0 (rag) . TragMax represents angular data at 
the lower edge assuming that the home position of the pan 
tiler 28 is 0 (rag) . TdatMin represents internal count data 
at the upper edge of the pan tilter controller 25. TdatMax 
represents internal count data at the lower edge of the pan 
tilter controller 25. 
[0075] 



To obtain the tilt angle (p with the tilt data t, 
since the following relation is satisfied, 

(TragMax - cp) : (TragMax - TragMin) = (TratMax 

- t) : (TdatMax - TdatMin) 

the tilt angle cp is expressed as follows. 

9 = TragMax - (TragMax - TragMin) x (TdatMax - 
t) / (TdatMax - TdatMin) 
[0076] 

Thus, the tilt data t is expressed as follows, 
t = TdatMax - (TragMax - cp) x (TdatMax - TdatMin) 
/ (TragMax - TragMin) 
[0077] 

To obtain the tilt data t-new with the tilt angle 
P, since the following relation is satisfied, 

(TragMax - P) : (TragMax - TragMin) = (TdatMax 

- t-new) : (TdatMax - TdatMin) 

the tilt data TNew is expressed as follows. 

T - new = TragMax - (TragMax - P) x (TdatMax - TdatMin) 
/ (TragMax - TragMin) 
[0078] 

Next, with reference to Fig. 12, a method for 
converting positional coordinates (£, T|) in the panorama 
operation area 6B into angular coordinates (a, P) of the pan 
tilter 28 and a method for converting angular information 
(0, cp) of the pan tilter 28 into positional coordinates (x, 
y) in the panorama operation area 6B will be described. In 
Fig. 12A, PragMin represents angular data at the left edge 



assuming that the home position of the pan tilter 28 is 0 
(rag) . PragMax represents angular data at the right edge 
assuming that the home position of the pan tilter 28 is 0 
(rag) . Ny 2 representsahorizontalcoordinateof thepanorama 
operation area 6B. -Ny 2 / 2 represents coordinate data at 
the left edge of the panorama operation area 6B . Ny 2 / 2 
represents coordinate data at the right edge of the panorama 
operation area 6B. 
[0079] 

To obtain the pan angle a with the coordinate data 
since the following relation is satisfied, 

(PragMax - a) : (PragMax - PragMin) = (Ny 2 / 2 

- §) : Ny 2 

the pan angle a is expressed as follows. 

a = PragMax - (PragMax - PragMin) x (Ny 2 / 2 - 

/ Ny 2 

[0080] 

To obtain the coordinate data x with the pan angle 
9, since the following relation is satisfied, 

(PragMax - 9) : (PragMax - PragMin) = (Ny 2 / 2 

- x) : Ny 2 

the coordinate data x is expressed as follows. 

x = Ny 2 / 2 - (PragMax - 9) x Ny 2 / (PragMax - PragMin) 
[0081] 

In Fig. 12B, TragMin represents angular data at 
the upper edge assuming that the home position of the pan 
tilter 28 is 0 (rag) . TragMax represents angular data at 



the lower edge assuming that the home position of the pan 
tilter 28 is 0 (rag) . Nz 2 represents a vertical coordinate 
of the panorama operation area 6B . -Nz 2 / 2 represents 
coordinate data at the upper edge of the panorama operation 
area 6B . Nz 2 / 2 represents coordinate data at the lower 
edge of the panorama operation area 6B . 
[0082] 

To obtain the tilt angle P with the coordinate 
data r\, since the following relation is satisfied, 

(TragMax - (3) : (TragMax - TragMin) = (Nz 2 / 2 

- T\) : Nz 2 

the tilt angle (3 is expressed as follows. 

P = TragMax - (TragMax - TragMin) x (Nz 2 / 2 - 

Tl) / Nz 2 

[0083] 

To obtain the coordinate data y with the tilt angle 
(p, since the following relation is satisfied, 

(TragMax - cp) : (TragMax - TragMin) = (Nz 2 / 2 

- y) : Nz 2 

the coordinate data y is expressed as follows. 

y = Nz 2 / 2 - (TragMax - 9) x nz 2 / (TragMax - TragMin) 
[0084] 

Next, with reference to Fig. 13, a method for 
converting angle-of -view information (\j/, a>) captured by the 
pan tilter 28 into angle-of -view information (s, t) of the 
frame 6C in the panorama operation area 6B will be described . 
Fig. 13A shows the current angle-of -view information (\|/, 



©) of the pan tilter 28. The angle-of -view information (\|/, 
©) is expressed as follows. 

(\|/, a)) = 1 / y x (v|/0, coo) 
At this point, (\|/0, coO) represent the horizontal angle of 
viewandtheverticalangleofviewatthewideedge . yrepresents 
the magnification of the lens assuming that the wide edge 
is defined as one magnification. 
[0085] 

As shown in Fig. 13B, PragMin represents angular 
data at the left edge assuming that the home position of 
the pan tilter 28 is 0 (rag) . PragMax represents angular 
data at the right edge assuming that the home position of 
the pan tilter 28 is 0 (rag) . Ny 2 represents a horizontal 
coordinate of the panorama operation area 6B. -Ny 2 / 2 
represents coordinate data at the left edge of the panorama 
operation area 6B . Ny 2 / 2 represents coordinate data at 
the right edge of the panorama operation area 6B. 
[0086] 

To obtain the horizontal angle of view s with the 
horizontal angle of view \|/, since the following relation 
is satisfied, 

\|/ : (PragMax - PragMin) = s : Ny 2 
horizontal angle of view s is expressed as follows. 

s = \|/ x Ny 2 / (PragMax - PragMin) 
[0087] 

In Fig. 13C, TragMin represents angular data at 
the lower edge assuming that the home position of the pan 



tilter 2 8 is 0 (rag) . TragMax represents angular data at 
the upper edge assuming that the home position of the pan 
tilter 28 is 0 (rag) . Nz 2 represents a vertical coordinate 
of the panorama operation area 6B. -Nz 2 / 2 represents 
coordinate data at the lower edge of the panorama operation 
area 6B . Nz 2 / 2 represents coordinate data at the upper 
edge of the panorama operation area 6B. 
[0088] 

To obtain the vertical angle of view t with the 
vertical angle of view oo, since the following relation is 
satisfied, 

oo : (TragMax - TragMin) = t : Nz 2 
the vertical angle of view t is expressed as follows, 
t = © x Nz 2 / (TragMax - TragMin) 
[0089] 

Thus, the angle-of -view information (s, t) shown 
in Fig. 13D is displayed as the frame 6C in the panorama 
operation area 6B . 
[0090] 

Next, with reference to Fig. 14, a method for 
converting the positional information (z) of the zoom lens 
16 into magnification information (y) will be described. 
In Fig . 14 , the vertical axis represents information of lens 
magnification, whereas the horizontal axis represents the 
internal information of zoom lens. The positional 
information (z) of the zoom lens 16 is converted into the 
magnification information (y) by thecomputer 1 corresponding 
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to a conversion graph shown in Fig. 14. For example, the 
positionalinf ormation (z) isconvertedintothemagnif ication 
information (y) corresponding to a ROM table or an equation . 
[0091] 

Next, with reference to Fig. 15, an example of 
a control algorithm of the computer 1 will be described. 
When the program is executed, the flow advances to step SI. 
At step SI, the operation area 6, the panorama operation 
area 6B, the cursor 7, and the pan filter limiter 6D are 
initialized and displayed on the monitor 2 as shown in Fig. 
2. The range of the pan tilter limiter 6D may be fixed or 
variable. At step S2, a timer is set so that the computer 
1 communicates with the mode controller 23 at predetermined 
intervals. After such initial setup operations have been 
completed, the flow advances to step S3. At step S3, the 
system waits for an occurrence of an event. Corresponding 
to an event that occurs, the flow advances to a relevant 
step (for example, a timer event (at step S4) , a mouse button 
down event (at step S5) , a mouse button up event (at step 
S6) , and a mouse move event (at step S7) ) . 
[0092] 

Next , with reference to a flow chart shown in Fig . 
16, the algorithm of the timer event will be described. The 
t imer event is anevent for causing the computer ltocommunicate 
with the mode controller 23 at predetermined intervals . The 
timer event occurs at intervals of for example 50 msec . When 
the timer event occurs, the flow advances to step Sll. At 



stepSll, the system determines whether or not a communication 
port has been set. When the communication port has been 
set (namely, the determined result at step Sll is Yes) , the 
flow advances to step S12. When the communication port has 
not been set (namely, the determined result at step Sll is 
No) , the flow advances to step S18. At the first time the 
communication port has not been set, the flow advances to 
step S18. At step S18, the system opens the communication 
port. Actually, at step S18, the system opens an RS-232C 
port of the computer 1. Thereafter, the flow advances to 
step S16. 

[0093] 

Thereafter, in the timer event , the systemperf orras 
a receive data checking process, an analyzing process, a 
data sending process for data stored in the send buffer (such 
as the drive command for the pan tilter 28) , and/or a 
communication data sending process for state check requests 
for the pan tilter 28 and the zoom lens 16 . In this algorithm, 
the flow advances from step Sll to step S12 . At step S12, 
the system determines whether or not data is stored in the 
receive buffer. When data is stored in the receive buffer 
(namely, the determined result at step S12 is Yes) , the flow 
advances to step S13 . When data is not stored in the receive 
buffer (namely, the determined result at step S12 is No) , 
thef lowadvancestostepS14 . AtstepS13, the systemanalyzes 
receivedatastoredinthereceivebuf f erandobtainspositional 
information (p, t) of the pan tilter 28 and positional 



information (z) of the zoom lens 16 that have been requested 
to the mode controller 23 . The system converts thepositional 
information (p, t) of the pan tilter 28 and the positional 
information (z) of the zoom lens 16 into angular information 
(0, (p) of the pan tilter 28 and magnification information 
(y) of the zoom lens 16 corresponding to methods shown in 
Fig. 11 and 14. 
[0094] 

At step S14, the system determines whether or not 
a data send request has been issued . When a data send request 
has been issued (Flag-so == True) (namely, the determined 
result at step S14 is Yes) , the flow advances to step S19. 
At step S19, the system sends data stored in the send buffer 
and resets the send request flag (Flag-so == False) . Next, 
the flow advances to step S16. An example of data stored 
in the send buffer is data of a drive command of the pan 
tilter 28 designated with the mouse 8 . When a send request 
has not been issued (Flag-so== False) (namely, the determined 
result at step S14 is No) , the flow advances to step S15. 
At step S15, the system sends position request commands for 
the pan tilter 2 8 and the zoom lens 16 from the computer 
1 to the mode controller 23 . 
[0095] 

At step S16 , the system compares the oldpositional 
information of the pan tiler 28 with the new positional 
information thereof and determines whether or not the 
positional information (p, t) hasvaried. Whenthepositional 
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information (p, t) of the pan tilter 28 has varied (namely, 
the determined result at step S16 is Yes) , the flow advances 
to step S20. When the positional information (p, t) of the 
pan tilter 28 has not varied (namely, the determined result 
at step S16 is No) , the flow advances to step S17. At step 
S17, the system compares the old positional information of 
the zoom lens 16 with the new positional information thereof 
and determines whether or not the positional information 

(z) has varied. When the positional information (z) of the 
zoom lens 16 has varied (namely, the determined result at 
step S17 is Yes) , the flow advances to step S20. When the 
positional information (z) of the zoom lens 16 has not varied 

(namely, the determined result at step S17 is No) , this event 
is completed. 
[0096] 

At step S20, when the positional information (p, 
t) of the pan tilter 28 and/or the positional information 
(z) of the zoom lens 16 has varied, the system redraws the 
frame 6C in the panorama operation area 6B . At this point, 
the system converts the positional information (p, t) of 
the pan tilter 28 into the angular information (0, cp) . In 
addition, the system converts the positional information 
(z) of the zoom lens 16 into the magnification information 
(y) . With the converted angular information (0, (p) and 
magnification information (y) , the system calculates 
positional coordinates (x, y) of the pan tilter 28 and 
angle-of -view information (s, t) that is the angle of view 
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displayed in the operation area 6A corresponding to Eq. (7) 
and Eq. (8) , respectively. Corresponding to the resultant 
positional coordinates (x, y) andangle-of -view information 
(s, t) , the system draws the frame 6C in thepanorama operation 
area 6B . 

[0097] 

At step S16 , the system compares the oldpositional 
information (p, t) of the pan tilter 28 with the newpositional 
information (p, t) thereof. Alternatively, the system may 
compare the old angular information (G, cp) of the pan tilter 
2 8 with the new angular information (9, (p) thereof. In this 
case, at step S20, with the new angular information (0, cp) , 
the system calculates the positional coordinates (x, y) 
corresponding to Eq. (7). Likewise, atstepS17, the system 
compares the oldpositional information (z) of the zoom lens 
16 with the new positional information (z) thereof. 
Alternatively, the system may compare the old magnification 
information (y) of the zoom lens 16 with the newmagnification 
information (y) thereof. Inthiscase, atstepS20, the system 
calculates the angular information (s, t) with the new 
magnification information (y) corresponding to Eq. (8) . 
[0098] 

Next , with reference to a flow chart shown in Fig . 
17, the algorithm of the mouse move event will be described. 
The mouse move event is an event that occurs when the mouse 
8 is moved. According to the present invention, the mouse 
move event is used to select a drive position of the pan 
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tilter28. When the mouse move event occurs , the f lowadvances 
to step S21. At step S21, the system determines whether 
or not the mouse pointer of the mouse 8 is present in the 
operation area 6A, the panorama operation area 6B, or the 
other area shown in Fig. 2. When the mouse pointer of the 
mouse 8 is present in the operation area 6A (namely, the 
determined result at step S21 is Yes) , the flow advances 
to step S22. When the mouse pointer of the mouse 8 is not 
present in the operation area 6A (namely, the determined 
result at step S21 is No) , the flow advances to step S24 . 
At stepS22, the system sets an operation area flag (Flag-rin 
= = True) and clears a panorama operation area flag (Flag-pin 
== False) . 

[0099] 

At step S24, since the mouse pointer of the mouse 
8 is not present in the operation area 6A, the system clears 
the operation area flag (Flag-rin == False) . At step S25, 
the system determines whether or not the mouse pointer of 
the mouse 8 is present in the panorama operation area 6B . 
When the mouse pointer of the mouse 8 is present in the panorama 
operation area 6B (namely, the determined result at step 
S25 is Yes) , the flow advances to step S26. When the mouse 
pointer of the mouse 8 is not present in the panorama operation 
area 6B (namely, the determined result at step S25 is No) , 
the flow advances to step S27. At step S26, the system sets 
the panorama operation area flag (Flag-pin == True) . At 
stepS2 7, since the mouse pointer of the mouse 8 is not present 
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inthepanoramaoperationarea6B, thesystemclearthepanorama 
operation area flag (Flag-pin == False) . 
[0100] 

When the mouse pointer of the mouse 8 is present 
in the operation area 6A or the panorama operation area 6B 
(namely, the determined result at step S21 or step S25 is 
Yes) , at step S23 , the system obtains positional coordinates 
(£, r|) of the mouse pointer of the mouse 8 assuming that 
the center of the operation area is defined as (0, 0) of 
relative coordinates. 
[0101] 

In this flow chart, when the mouse pointer of the 
mouse 8 is present in the operation area 6A (namely, the 
determined result at step S22 is Yes) , the system sets the 
operationareaf lag (Flag-rin==True) . Whenthemousepointer 
of the mouse 8 is not present in the operation area 6A (namely, 
the determined result at step S22 is No) , the system clears 
the operation area flag (Flag-rin == False) . When the mouse 
pointer of the mouse 8 is present in the panorama operation 
area 6B (namely, the determined result at step S25 is Yes) , 
the system sets the panorama operation area flag (Flag-pin 
== True) . When the mouse pointer 8 is not present in the 
panorama operation area 6A (namely, the determined result 
at step S25 is No) , the system clears the panorama operation 
area flag (Flag-pin == False) . When the mouse pointer of 
the mouse 8 is present in the operation area 6A or the panorama 
operation area 6B (namely, the determined result at step 



S21 or S35 is Yes) , the system designates the positional 
coordinates of the mouse pointer of the mouse 8 to ti) 
assuming that the center of each operation area is defined 
as (0, 0) of relative coordinates. 
[0102] 

Next, the mouse button down event and the button 
up event will be described. In the method for directly 
designating a desired point of the operation area 6A or the 
panorama operation area 6B, only the algorithm of a mouse 
button down event shown in Fig. 18 is used. In the method 
for designating a desired point generated with a desired 
area, both the algorithm of a mouse button down event shown 
in Fig. 19 and the algorithm of a mouse button up event shown 
in Fig. 20 are used. 
[0103] 

With reference to a flow chart shown in Fig. 18, 
the algorithm of the button down event for the method for 
directly designating a desired point of the operation area 
will be described. This event is an event that occurs when 
the left button of the mouse 8 is pressed. In the present 
invention, this event is used as trigger information for 
driving the pan tilter 28. When this event occurs, the flow 
advances to step S31. At step S31, the system determines 
whether or not the mouse pointer 8 is present in the operation 
area 6A corresponding to the operation area flag. When the 
operation area flag has been set (Flag-rin == True) (namely, 
the determined result at step S31 is Yes) , since the mouse 



pointer of the mouse 8 is present in the operation area 6A, 
the flow advances to step S32 . When the operation area flag 
has been cleared (Flag-rin == False) (namely, the determined 
result at step S31 is No) , since the mouse pointer of the 
mouse 8 is not present in the operation area 6A, the flow 
advances to step S34 . 
[0104] 

When the mouse pointer of the mouse 8 is present 
in the operation area 6A (namely, the determined result at 
step S31 is Yes) , the flow advances to step S32. At step 
S32, the system calculates angular information (a, P) of 
the pan tilter 28 with the angular information (0, (p) of the 
current pan tilter 28 obtained from the received data, the 
magnification information (y) of the zoom lens 16, and the 
positional coordinate (£, T]) of the mouse pointer of the 
mouse 8 in the operation area 6A corresponding to Eq. (4) 
or Eq. (5) so that the designated object in the operation 
area is placed at the center of the screen. 
[0105] 

At step S33, the system converts the angular 
information (a, 0) of the pan tilter 28 into the internal 
positional information (p-new, t-new) corresponding to the 
method shown in Fig. 11. The system stores the converted 
positional information (PNew, TNew) in the send buff er along 
with the absolute position drive command of the pan tilter 
28. In addition, the system sets the data send request flag 
(FlagSo == True) and sends the data with the process of the 



timer event. 
[0106] 

After the system has determined that the mouse 
pointer of the mouse 8 is not present in the operation area 
5 6A (namely, the determined result at step S31 is No) , the 

f low advances to step S34 . AtstepS34, the system determines 
whether or not the mouse pointer of the mouse 8 is present 
in thepanoramaoperationarea6B corresponding to thepanorama 
operation area flag. When the panorama operation flag has 

10 been set (Flag-pin == True) (namely, the determined result 

at step S34 is Yes) , since the mouse pointer of the mouse 
8 of the panorama operation area 6B is present in the panorama 
operation area 6B, the flow advances to step S35. When the 
panorama operation flag has been cleared (Flag-pin == Fale) 

15 (namely, the determined result at step S34 is No) , this event 

is completed. 
[0107] 

In this flow chart, the system determines whether 
or not the mouse pointer of the mouse 8 is present in the 
2 0 operation area 6A or the panorama operation area 6B 

corresponding to the operation area flag (Flag-rin) and the 
panorama operation area flag (Flag-pin) . When the mouse 
pointer of the mouse 8 is not present in the operation area 
6A and the panorama operation area 6B, this event becomes 
25 invalid. 

[0108] 

When the mouse pointer of the mouse 8 is present 
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in the panorama operation area 6B (namely, the determined 
result at step S34 is Yes) , the flow advances to step S35. 
At step S35, the system calculates angular information (a, 
(3) of the pan tilter 28 with the positional information {E,, 
r\) at the mouse pointer of the mouse 8 in the panorama operation 
area 6B corresponding to Eq. (9) so that the designated obj ect 
in the operation area is placed at the center of the screen. 
Thereafter, the flow advances to step S33 . 
[0109] 

Next, with reference to Figs. 19 and 20, the 
algorithms of the button down event and the button up event 
for the method for designating a desired point generated 
with a desired area in the panorama operation area 6B will 
be described, respectively. 
[0110] 

With reference to the flow chart shown in Fig. 
19, the algorithm of the button down event will be described. 
This event is an event that occurs when the left button of 
the mouse 8 is pressed. In this embodiment, this event is 
used as an event for determining the start point of a desired 
area. When this event occurs, the flow advances to step 
S41. At step S41, the system determines whether or not the 
mouse pointer of the mouse 8 is present in the operation 
area 6A corresponding to the operation area flag (Flag-rin) . 
When the operation area flag has been set (Flag-rin == True) 
(namely, the determined result at step S41 is Yes) , since 
the mouse pointer of the mouse 8 is present in the operation 



area 6A, the flow advances to step S42 . When the operation 
area flag has been cleared (Flag-rin == False) (namely, the 
determined result at step S41 is No) , since the mouse pointer 
of the mouse 8 is not present in the operation area 6A, the 
flow advances to step S44 . 
[0111] 

When the mouse pointer of the mouse 8 is present 
in the operation area 6A (namely, the determined result at 
step S41 is Yes) , at step S42, the system sets an operation 
areastartpointobtainf lag (Flag-rstart =True) . Thereafter, 
the flow advances to step S43 . At step S43 , the system stores 
positional coordinates (ml, nl) at which the left button 
of the mouse 8 is pressed as the start point of the desired 
area . 

[0112] 

After the system has determined that the mouse 
pointer of the mouse 8 is not present in the operation area 
6A, at step S44, the system determines whether or not the 
mousepointerof themouse8 ispresent in thepanorama operation 
area 6B corresponding to the panorama operation area flag 
(Flag-pin) . When the panorama operation area flag has been 
set (Flag-pin == True) (namely, the determined result at 
step S44 is Yes) , since the mouse pointer of the mouse 8 
ispresent in the panorama operation area 6B, the f lowadvances 
to step S45 . When the panorama operation area flag has been 
cleared (Flag-pin == False) (namely, the determined result 
at step S44 is No) , this event is completed. 
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[0113] 

In this flow chart, the system determines whether 
or not the mouse pointer of the mouse 8 is present in the 
operation area 6A or the panorama operation area 6B 
corresponding to the operation area flag (Flag-rin) and the 
panorama operation area flag (Flag-pin) . When the mouse 
pointer of the mouse 8 is not in the operation area 6A and 
the panorama operation area 6B, this event becomes invalid. 
[0114] 

When the mouse pointer of the mouse 8 is present 
in the panorama operation area 6B (namely, the determined 
result at step 44 is Yes) , the flow advances to step S45. 
At step S45, the system sets a panorama operation area start 
point obtain flag (Flag-pstart) . Thereafter, the flow 
advances to step S43 . 
[0115] 

Next, with reference to a flow chart shown in Fig. 
20, the algorithm of the button up event will be described. 
This event is an event that occurs when the left button of 
the mouse 8 is released. In the present invention, the button 
up event is used as an event for determining the end point 
of a desired area. 
[0116] 

When this event occurs, the flow advances to step 
S51. At step S51, the system determines whether or not the 
operation area flag has been set (Flag-rin == True) (namely, 
the mouse pointer of the mouse 8 is present in the operation 
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area 6A) . When the operation area flag has been set (Flag-rin 
= True) (namely, the determined result at step S51 is Yes) , 
since the mouse pointer of the mouse 8 is present in the 
operation area 6A, the flow advances to step S52 . When the 
operation area flag has been cleared (Flag-rin == False) 
(namely, the determined result at step S51 is No) , since 
the mouse pointer of the mouse 8 is not present in the operation 
area 6A, the flow advances to step S57. At step S52, the 
system determines whether or not the left button of the mouse 
8 has been pressed in the operation area 6A corresponding 
to an operation area start point obtain flag (Flag-rstart) . 
When the start point obtain flag has been set (Flag-rstart 
== True) (namely, the determined result at step S52 is Yes) , 
since the left button of the mouse 8 has been pressed in 
the operation area 6A, the flow advances to step S53 . When 
the start point obtain flag has been cleared (Flag-rstart 
= = False) (namely, the determined result at step S52 is No) , 
since the left button of the mouse 8 has not been pressed 
in the operation area 6A, the flow advances to step S57. 
[0117] 

In other words, at steps S51 and S52, the system 
determines whether or not the operation area flag and the 
operat ionarea st art point obtainf laghavebeen set or cleared. 
When the operation area flag and the start point obtain flag 
have been set (Flag-rin == True and Flag-rstart == True) , 
the system determines that the drive command has taken place 
in the operation area 6A. Otherwise, at steps S57 and S58, 



the system determines whether or not the panorama operation 
area flag (Flag-pin) and the panorama operation area start 
point obtain flag (Flag-pstart) have been set or cleared. 
[0118] 

When the drive command has taken place in the 
operation area (namely, the operation area flag and the start 
pointobtainf laghavebeenset ( Flag- rin==TrueandFlag-r start 
== True), at step S53, the system stores the positional 
coordinates (m2, n2) at which the left button of the mouse 
8 has been released in the operation area 6A as the end point 
of the desired area. Thereafter, the system calculates 
positional information (^, n) as the coordinates of the center 
of therectangleareageneratedwiththepos it ional coordinates 
(ml , nl) of thestartpointof thedesiredareaandthepositional 
coordinates (m2, n2) of the end point thereof. 
[0119] 

At step S54, the system calculates angular 
information (a, (3) of the pan tilter 28 with the angular 
information (9, cp) of thepan tilter obtained from the received 
data, the magnification information (y) of the zoom lens 
16, and the positional information (£, ll) at the mouse pointer 
of the mouse 8 corresponding to Eq. (4) or Eq. (5) . 
[0120] 

At step S55, the system converts the angular 
information (a, (3) of the pan tilter 28 into the internal 
positional information (p-new, t-new) of the pan tiler 28 
corresponding to the method shown in Fig. 11 and stores the 
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positional information (p-new, t-new) to the send buffer 
along with the absolute posit ion drive command. In addition, 
the system sets the data send request flag (Flag-so == True) 
and sends data with the process of the timer event. 
[0121] 

At step S56, after the system has checked the mouse 
button up event in each operation area, the system clears 
the operation area start point obtain flag and the panorama 
operation area start point obtain flag (Flag-rstart == False 
and Flag-pstart == False) . Thereafter, this event is 
completed. 

[0122] 

At step S57, the system determines whether or not 
the mouse pointer of the mouse 8 is present in the panorama 
operation area 6B corresponding to the panorama operation 
area flag (Flag-pin) . When the panorama operation area flag 
hasbeenset (Flag-pin== True) (namely, the determined result 
at step S57 is Yes) , since the mouse pointer of the mouse 
8 is present in the panorama operation area 6B, the flow 
advances to step S58 . When the panorama operation area flag 
has not been set (Flag-pin == False) , since the mouse pointer 
of the mouse 8 is not present in the panorama operation area 
6B, the flow advances to step S56. At step S58, the system 
determines whether or not the left button of the mouse 8 
hasbeenpressedinthepanoramaoperationarea6Bcorresponding 
to the panorama operation area start point obtain flag 
(Flag-pstart) . When the start point obtain flag has been 



set (Flag-pstart == True) (namely, the determined result 
at step S58 is Yes) , since the left button of the mouse 8 
has been pressed in the panorama operation area 6B, the flow 
advances to step S59. When the start point obtain flag has 
not been set (Flag-pstart == False) (namely, the determined 
result at step S58 is No) , since the left button of the mouse 
8 has not been pressed in the panorama operation area 6B, 
the flow advances to step S56. 
[0123] 

When the panorama operation area flag and the 
panorama operation start point obtain flag have been set 
(FlagPin == True and FlagPstart == True) at steps S57 and 
S58, the system determines that a drive command has issued 
in the panorama operation area 6B . When the conditions at 
steps S51, S52, and S58 are not satisfied, this event becomes 
invalid. 

[0124] 

Whenthedrivecommandhasbeenissuedinthepanorama 
operation area 6B (namely, the panorama operation area flag 
and the start obtain flag have been set (Flag-pin == True 
and Flag-pstart == True), the flow advances to step S59. 
At step S59, the system stores the positional coordinates 
(m2, n2) at which the left button of the mouse 8 has been 
released in the panorama operation area 6B as the end point 
of the desired area. The system calculates the positional 
information (Z,, t\) of the mouse pointer of the mouse 8 as 
the coordinates of the center of the rectangle area with 



the positional coordinates (ml, nl) of the start point of 
the desired area that has been stored and the positional 
coordinates (m2, n2) of the end point of the desired area 
corresponding to Eq. (6) . 
[0125] 

At step S60, the system calculates angular 
information (a, P) of the pan tilter 28 with the positional 
information (^, T|) at the mouse pointer of the mouse 8 in 
the panorama operation area 6B corresponding to Eq. (9) so 
that the designated object in the panorama operation area 
is placed at the center of the screen. Thereafter, the flow 
advances to step S55. 
[0126] 

In the first embodiment, whenever the pan tilter 
camera 3 sends a picture to the computer 1, the computer 
1 combines it anddisplays the combinedpicture inthe panorama 
operation area 6B. Alternatively, after the computer has 
combined all pictures, it may display the resultant picture 
in the panorama operation area 6B. 
[0127] 

According to the first embodiment, the computer 
1 forms a panorama picture. Alternatively, the server 9 
may form a panorama picture and send the data of the panorama 
picture to the computer 1 through a low capacity network 
line, and display the picture in the panorama operation area 
6B of the monitor 2 . 



According to the first embodiment, the operation 
area 6A and the panorama operation area 6B are displayed 
on the monitor 2 connected to the computer 1 . Alternatively, 
the operation area 6A and/or the panorama operation area 
6B may be displayed on another display unit other than the 
monitor 2 . 

[0129] 

According to the first embodiment, the pan tilter 
camera 3 is driven by operating the operation area 6A and 
thepanoramaoperationarea6Bwiththemouse8 . Alternatively, 
one of the operation area 6A and the panorama operation area 
6B may be operated with the mouse 8. 
[0130] 

According to the first embodiment, the operation 
area 6A and the panorama operation area 6B are displayed 
onthemonitor 2 . Alternatively, only the panorama operation 
area 6B may be displayed on the monitor 2 . 
[0131] 

According to the first embodiment, the operation 
area 6A and the panorama operation area 6B are displayed 
on the monitor 2 . By operating the operation area 6A and 
the panorama operation area 6B with the mouse 8, the pan 
tilter camera 3 is freely driven. Alternatively, a panorama 
picture may be displayed on the monitor 2. In this case, 
the pan tilter camera 3 maybe driven with an operationportion 
such as eight -direction keys. 
[0132] 



Accordingtothesecondembodiment , f orsimplicity , 
one computer 1 is connected to the remote server 9 and the 
pan tilter camera 3 that are disposed at a remote place. 
Alternatively, a plurality of servers 9 and a plurality of 
pan tilter cameras 3 maybe disposed worldwide . For example, 
one pan tilter camera 3 may be controlled by a plurality 
of computers through Internet, for example. 
[0133] 

According to the above -described embodiments, the 
photographing range of the pan tilter camera 3 may be the 
maximum moving range of the pan tilter camera 3 or limited 
with a limiter. The function for limited the photographing 
range with the limiter may be provided by the pan tilter 
camera 3 or the computer 1 . 
[0134] 

In the first embodiments , a desiredpoint generated 
with a desired area is placed at the center of thereof. 
Alternatively, a desired point may be placed at for example 
the center of gravity, the incenter, the circumcenter , or 
the orthocenter of the area. 
[0135] 

According to the first embodiment, a panorama 
picture displayed in the panorama operation area 6B is not 
limited as long as it represents the environment in which 
thepantilter camera3 isdisposed. Forexample, thepanorama 
picturemaybeamovingpicture, anintermittentstillpicture, 
or a still picture. 
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[0136] 

[Effects of the Inveniton] 

Accordingtothepresent invent ion, adesiredobj ect 
can be displayed at the center of the operation area with 
the clicking operation of the mouse. In addition, since 
the position to which the pan tilter moves can be predicted 
beforehand, on a communication line that causes picture and 
information data to be delayed and/or lost (suchas Internet) , 
the user can seamlessly operate the pan tilter camera . Thus , 
according to the present invention, the pan tilter camera 
can be easily operated with high visibility. 
[0137] 

Accordingtothepresent invention, withapanorama 
picture, the user can see the environment in which the 
photographing apparatus is disposed at a glance. Since the 
positional information of the pan tilter, the angle of view 
of the zoom lens, and the moving range of the pan tilter 
are added as information to the picture, the user can easily 
know the state of the photographing apparatus . In addition, 
when the user designates a desired object in the panorama 
operation area, he or she can easily capture it in the field 
of view of the picture to be photographed. Moreover, by 
designating an object in the operation area, the user can 
precisely adjust the position that cannot be designated in 
the panorama operation area. 
[Brief Description of the Drawings] 
[Fig. 1] 
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An external view for explaining a system according 
to the present invention. 
[Fig. 2] 

A schematic diagram for explaining a screen 
5 according to the present invention. 

[Fig. 3] 

Ablock diagram showing the structure of the system 
according to the embodiment of the present invention. 
[Fig. 4] 

10 A schematic diagram for explaining a method for 

generating a panorama picture according to the present 
invention. 

[Fig. 5] 

A schematic diagram for explaining a method for 
15 generating a panorama picture according to the present 

invention. 

[Fig. 6] 

A schematic diagram for explaining a method for 
generating a panorama picture according to the present 
2 0 invention. 

[Fig. 7] 

A schematic diagram for explaining a method for 
generating angular information of a pan tilter camera with 
positional coordinates inapanoramaoperationareaaccording 
25 to the present invention. 

[Fig. 8] 

A schematic diagram for explaining a plane - 
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spherical surface converting method according to the present 
invention. 

[Fig. 9] 

A schematic diagram for explaining a coordinate 
5 converting method in the operation area according to the 

present invention. 
[Fig. 10] 

A schematic diagram for explaining a coordinate 
converting method in the panorama operation area according 
10 to the present invention. 

[Fig. 11] 

A schematic diagram for explaining positional 
information and angular information of a pan tilter camera 
according to the present invention. 
15 [Fig. 12] 

A schematic diagram for explaining angular 
coordinatesof thepantiltercameraandpos it ional coordinates 
in the panorama operation area according to the present 
invention. 
20 [Fig. 13] 

A schematic diagram for explaining the angle of 
view of the pan tilter camera and a frame in the panorama 
operation area according to the present invention. 
[Fig. 14] 

25 A graph for explaining a conversion method of zoom 

dataandmagnif i cat iondataaccordingtothepresent invent ion . 
[Fig. 15] 
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A flow chart showing an example of the overall 
process according to the present invention. 
[Fig. 16] 

A flow chart showing an example of the process 
of a timer event according to the present invention. 
[Fig. 17] 

A flow chart showing an example of the process 
of a mouse moving event according to the present invention. 
[Fig. 18] 

A flow chart showing an example of the process 
of amousebuttondowneventaccordingtothepresent invention. 
[Fig. 19] 

A flow chart showing another example of the process 
of amousebutton down event accordingtothepresent invention. 
[Fig. 20] 

A flow chart showing an example of the process 
of a mouse up/down event according to the present invention. 
[Fig. 21] 

A system through the conventional transmission 

path. 

[Description of Reference Numerals] 

1 . . . Computer, 2 . . . Monitor, 6A . . . Operation area, 6B . . . 
Panorama operation area, 7 . . . Cursor, 9 . . . Server, 11 ... 
Camera portion, 12 . . . Pan tilter portion, 13 . . . TV monitor, 
14 . . . Pointing device, 15 . . . Lens block, 16 . . . Zoom lens, 
17 ... Zoom portion, 18, 26, 27 ... Motor, 19 ... Solid 
state imagepickupdevice, 20 . . . Signal separating/automatic 
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gain adjusting circuit, 21 . . . A/D converter, 22 . . . Signal 
processing, 23 . . . Mode controller , 24 . . . Camera controller , 
25 . . . Pan tilter controller, 28 . . . Pan tilter, 29 . . . Video 
capture, 30 . . . Storing portion, 31 . . . Controlling portion 
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[Title of Document] Abstract 
[Abstract] 

[Subject] 

To easily obtain a desired picture by displaying 
a part or whole of the moving range of a pan tilter camera 
that is connected through a transmission path as a panorama 
picture and designating a desired position or a desired area 
on the panorama picture. 
[Solving Means] 

A picture photographed by a camera portion 11 is 
sent to a video capturing portion 29 of a computer 1. The 
picture is displayed in an operation area 6A of a monitor 
2 . A panorama picture of which pictures in part or all moving 
range of a pan tilter 28 are combined is displayed in a panorama 
operation area 6B. A pan tilter portion 12 sends positional 
information of pan and tilt to the computer 1 through a mode 
controller 23. With a mouse 8, the operation area 6A and 
6B are operated so as to the computer 1 obtain data for driving 
the pan tilter 28. Thus, the selected object is displayed 
at the center of the operation area. 
[Selected Drawing] Fig. 3 
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Translation of Drawings 



FIG . 
3 . . 



PAN TILTER CAMERA 
SERVER 



OPERATION AREA 
PANORAMA OPERATION AREA 



FIG. 

(A) 

(B) 

(C) 

(D) 

(E) 

15 . 

22 . 



28 . 
26 . 



23 . 
2 . . 



VIDEO SIGNAL 

DRIVE COMMAND 

POSITIONAL INFORMATION 

CONTROL, POSITIONAL INFORMATION 

ZOOM 
LENS BLOCK 
SIGNAL PROCESSING 
CAMERA PORTION 
TV MONITOR 
PAN TILTER 
PAN MOTOR 
TILT MOTOR 
CAMERA CONTROLLER 
PAN TILTER CONTROLLER 
MODE CONTROLLER 
MONITOR 
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OPERATION AREA 

PANORAMA OPERATION AREA 
SERVER 

CONTROLLING PORTION 

VIDEO CAPTURE 
STORING PORTION 
COMPUTER 
. POINTING DEVICE 



FIG. 6 

(1) ... PICTURE CONVERTED INTO SPHERICAL SURFACE IS 
NORMALIZED WITH LATITUDE AND LONGITUDE 

(2) ... CAPTURED PICTURE 

(3) ... OBTAINED PICTURE 

FIG. 7 

(1) ... PAN ANGLE 

(2) ... COORDINATE DATA 

(3) ... TILT ANGLE 

(4) ... COORDINATE DATA 



FIG. 11A 

(1) ... PAN ANGLE 

(2) ... PAN DATA 

FIG. 11B 

(1) ... TILT ANGLE 
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(2) 



TILT DATA 



FIG. 12A 

(1) ... PAN ANGLE 
5 (2) . . . COORDINATE DATA 



FIG. 12B 

(1) ... TILT ANGLE 

(2) ... COORDINATE DATA 

FIG. 13A 

(1) ... HORIZONTAL ANGLE OF VIEW (\|/) 

(2) ... VERTICAL ANGLE OF VIEW (©) 



15 FIG. 13B 

(1) ... PAN ANGLE 

(2) ... COORDINATE DATA 



FIG. 13C 
20 (1) . . . PAN ANGLE 

(2) ... COORDINATE DATA 



FIG. 13D 

(1) ... HORIZONTAL ANGLE OF VIEW (s) 
25 (2) ... VERTICAL ANGLE OF VIEW (t) 

FIG. 14 

(1) ... INFORMATION OF LENS MAGNIFICATION 
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(2) 



INTERNAL INFORMATION OF ZOOM LENS 



FIG. 15 
(A) ... START 
5 SI . . . INITIALIZE OPERATION AREA ON MONITOR. 

52 ... SET TIMER EVENT. 

53 ... DETERMINE TYPE OF EVENT THAT OCCURS. 

54 ... TIMER EVENT 

55 ... MOUSE BUTTON DOWN EVENT 
10 S6 . . . MOUSE BUTTON UP EVENT 

S7 ... MOUSE MOVE EVENT 

FIG. 16 

(A) ... START 

15 (B) ... NO 

(C) ... YES 

(D) ... NO 

(E) ... YES 

(E) ' ... YES (TRUE) 
20 (E) " ... NO (FALSE) 

(F) ... YES 

(G) ... NO 

(H) ... YES 

( I ) ... NO 
25 (J) ... END 

511 . . . HAS COMMUNICATION PORT BEEN SET ? 

512 ... IS DATA STORED IN RECEIVE BUFFER ? 
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513 . . . ANALYZE RECEIVE DATA, OBTAIN PAN TILTER POSITION 
(p, t) AND ZOOM POSITION (z) , AND CALCULATE PAN TILTER ANGLE 
(9, (p) AND ZOOM MAGNIFICATION (y) . 

514 ... IS SEND REQUEST FLAG Flag-So TRUE ? 

5 S15 . . . SEND PAN TILER, ZOOM POSITION REQUEST COMMUNICATION 

DATA. 

51 8 ... SET COMMUNICATION PORT. 

519 . . . SEND DATA STORED IN SEND BUFFER AND RESET Flag-So 
TO FALSE. 

10 S16 . . . HAS PAN TILTER DATA VARIED ? 

S17 ... HAS ZOOM DATA VARIED ? 

S2 0 ... REDRAW FRAME IN PANORAMA PICTURE AREA. 



FIG. 17 

(A) ... START 

(B) ... NO 

(C) ... YES 

(D) ... NO 

(E) ... YES 

(F) ... END 

S21 ... IS MOUSE POINTER PRESENT IN OPERATION AREA ? 

S23 . . . OBTAIN COORDINATES T|) OF POSITIONOFMOUSE POINTER 
IN OPERATION AREA. 

S25 . . . IS MOUSE POINTER PRESENT IN PANORAMA OPERATION AREA? 
FIG. 18 

(A) ... START 

(B) ... NO (FALSE) 
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(C) . 


. YES (TRUE) 


(D) . 


. NO (FALSE) 


(E) . 


. YES (TRUE) 


(F) . 


. END 


S32 . 


. OBTAIN (a, P) WITH EACH DATA 



S33 . . . CONVERT (a, p) INTO PAN TILTER POSITIONAL COORDINATES 
(P-New, T-New) , STORE THEMAS ABSOLUTE POSITIONDRIVE REQUEST 
COMMUNICATION DATA TO COMMUNICATION BUFFER, AND SET Flag-So 





TO TRUE 




10 


S35 . . . 


OBTAIN (a, P) WITH EACH DATA. 




FIG. 19 






(A) ... 


START 




(B) ... 


NO (FALSE) 


15 


(C) ... 


YES (TRUE) 




(D) ... 


NO (FALSE) 




(E) ... 


YES (TRUE) 




(F) ... 


END 




S42 . . . 


SET START POINT OBTAIN FLAG Flag-Rstart TO TRUE 


20 


S43 . . . 


OBTAIN COORDINATES (ml, nl) AT THE POINT. 




S45 ... 


SET START POINT OBTAIN FLAG Flag- Ps tart TO TRUE 




FIG. 20 






(A) ... 


START 


25 


(B) ... 


NO (FALSE) 




(C) ... 


YES (TRUE) 




(D) . . . 


NO (FALSE) 
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(E) YES (TRUE) 

(F) ... NO (FALSE) 

(G) ... YES (TRUE) 

(H) ... NO (FALSE) 

(I) ... YES (TRUE) 
(J) ... END 

553 ... OBTAIN COORDINATES (m2, n2 ) AND THEREBY (§, T|) AT 
THIS POINT. 

554 ... OBTAIN (a, p) WITH EACH DATA. 

555 . . . CONVERT (a, 0) INTO PAN TILTER POSITIONAL COORDINATES 
(P-New, T-New) , STORE THEMAS ABSOLUTE POSITIONDRIVE REQUEST 
COMMUNICATION DATA TO COMMUNICATION BUFFER, AND SET Flag-So 
TO TRUE. 

556 . . . SET START POINT OBTAIN FLAG Flag-RStart TO FALSE 
AND Flag-Pstart TO FALSE. 

559 ... OBTAIN COORDINATES (m2 , n2) AND THEREBY (§, r\) AT 
THIS POINT. 

560 ... OBTAIN (a, P) WITH EACH DATA. 
FIG. 21 

3 ... PAN TILTER CAMERA 
9 . . . SERVER 
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